As more and more people have an interest in multimedia today, methods and systems are being developed in the field of hardware which will become the platform of multimedia. This type of hardware is designed to implement complex and sophisticated image processing in order to satisfy the needs of program software. Especially, in the video entertainment systems, the demand for image processing ability having more realistic and dramatic effects has been increased to satisfy the user's growing need.
The prior image processing technique is as follows. According to the prior art of the image processing system used in a video entertainment system, the background image where the surface of the land or the sea, the sky, or the space is displayed is combined with the foreground image where the character of the game is displayed, and the composite image is displayed on the monitor screen of the display. When a game player plays the game, the foreground image or background image is moved to show the motion of the character. That is, the foreground image is moved or rotated with the background image fixed or, conversely, the background image is moved or rotated with the foreground image fixed. The latter function is called the scroll function.
In addition to the scroll function described above, an image processing system has other functions such as the window function and the priority function. The following describes these functions.
Window function and priority function!
The following is a brief description of the window function and the priority function. Recently, more and more data has to be displayed on one monitor screen. To meet this need, the window function has been introduced; it divides the screen into one or more transparent image areas, called windows, and displays different types of data in each window. Windows are normally overlapped with the background image or with other windows. And, among a plurality of overlapping windows, only one window is displayed on top of the others according to the specified priority. This priority is determined by the priority function. The priority function plays an important role in today's image processing technique; that is, it not only determines the priority when windows are displayed and displays windows but also determines the priority between the background image and the fore ground image or the priority among a plurality of foreground images of various types.
Graphical image displayed by the scroll function!
The following is a description of the scroll function. The scroll function moves the background image while keeping the foreground image, containing the character, near the center of the display screen. Moving the background image does not mean the transformation of image information (that is, the background image itself is not moved); the background image is a virtual static image stored in the image memory of the image processing system. More precisely, we can say that a display screen frame or a player's viewpoint simply moves around the whole background image stored in an image memory as a result of actual calculation. Strictly speaking, a display screen is a virtual screen set up for each circuit which performs movement transformation or rotation transformation. However, in the following discussion, a display screen may be thought of as the monitor screen of a display because it is the screen on which images are displayed.
Scroll screen configuration!
There are two types of scroll screen formats: the cell format used on a home video entertainment system and the bit mapped format used on a personal computer. To display a cell format scroll screen on the display screen, a plurality of cell patterns, the same displayed as necessary, for example, each consisting of 8.times.8 pixels are combined and displayed on the display screen as background image data. The pattern data of a cell-image and the display screen location of a cell image are specified by data called pattern name data.
Display control of the scroll screen on a video entertainment system!
Normally, a video entertainment system having an image processing system has a CPU which controls the whole video entertainment system as well as image memory which contains image information. Pattern name data and cell-image pattern data are stored in video RAM (hereafter called VRAM) which is used as image memory. That is, when a video entertainment system displays a background image, VRAM contains image information, which has been read from a cassette ROM or a CD-ROM under control of the CPU, or image information which has been processed by the CPU. The video entertainment system first reads pattern name data from VRAM and, based on the pattern name data that has been read, accesses VRAM again to read the pattern data of the cell image to display the background image on the display screen (more specifically, on the monitor screen).
There are two types of scroll screens: one is a normal scroll screen which is moved up or down and right or left, and the other is a rotation scroll screen which is rotated. On the scroll screen, it is possible to predict pattern data and pattern name data to be used and, therefore, it is possible to access pattern data and pattern name data which are stored beforehand in VRAM. On the rotation screen, however, it is impossible to predict pattern data and pattern name data to be used. This means that pattern data and pattern name data must be stored in VRAM on a dot basis.
Relation among a character, rotation scroll screen, and VRAM!
The image data (called pattern data) of a character (for example, an airplane in a flight simulator) to be displayed in the foreground is stored in VRAM on a dot basis. Thus, to display the character on the display screen, the video entertainment system accesses VRAM, one dot at a time.
In this case, to superimpose a character (foreground) on the rotation scroll screen (background), it is necessary to access VRAM to read both pattern name data and pattern data, one dot at a time. This means that VRAM for storing pattern name data must be physically independent of VRAM for storing pattern data. More specifically, a video entertainment system with two rotation scroll screens must have a total of four VRAM areas: two VRAM areas for storing pattern name data and two VRAM areas for storing pattern data.
Calculation of standard scroll screen display control!
To move and scroll the scroll screen that is used as the background image, the following method is used:
1 From the movement rotation formula, calculate the X and Y coordinates of the starting point (leftmost end) of each line in the background image and the horizontal increments .DELTA. X and .DELTA. Y.
2 Add the horizontal increments .DELTA. X and .DELTA. Y to the X and Y coordinates of the start point of each line on a dot basis.
3 Calculate the position coordinates of each pixel.
4 Generate the image data in the background image by accessing image memory to get the address corresponding to this position coordinates.
In practice, screen coordinates are calculated under program control as shown below to find the starting coordinates (GXst, GYst) and the horizontal coordinate increments .DELTA. GX and .DELTA. GY. ##EQU1## A, B, C, D, E, F: Rotation matrix parameters Px, Py, Pz: Viewpoint coordinates
Xp, Yp, Zp: Viewpoint coordinates after rotation transformation PA1 Xs, Ys, Zs: Screen coordinates after rotation transformation PA1 Xst, Yst, Zst: Start coordinates on the display screen PA1 .DELTA. Xst, .DELTA. Yst: Vertical coordinate increment on the display screen PA1 .DELTA. X, .DELTA. Y: Horizontal coordinate increment on the display screen PA1 Vcnt: V count value PA1 (monitor screen vertical coordinate value) EQU GXst=k(Xs-Xp)+Xp EQU GYst=k(Ys-Yp)+Yp EQU .DELTA. GX=k.times.dX EQU .DELTA. GY=k.times.dY PA1 1 a plurality of signal processing means for outputting transformation processing signals necessary for executing transformation processing on the image information; PA1 2 a switching means, connected to a plurality of the signal processing means, for selecting and outputting one of the transformation processing signals according to a pre-set display switching signal; and PA1 3 an address generation means, connected to the switching means, for generating a read address corresponding to image information which has been transformed based on the transformation processing signal sent from the switching means. PA1 1 a plurality of signal processing means for outputting transformation processing signals necessary for executing transformation processing on the image information according to provided information under control of the CPU; PA1 2 a switching means, connected to a plurality of the signal processing means, for selecting and outputting one of the transformation processing signals according to a pre-set display switching signal; and PA1 3 an address generation means, connected to the switching means, for generating a read address corresponding to image information which has been transformed according to the transformation processing signal. PA1 1) a first video processor, connected to the bus line, for controlling the generation of sprite image data; PA1 2) a first video RAM, connected to the first video processor, for storing image information for generating the sprite image data; and PA1 3) a frame buffer for expanding the sprite image data, and PA1 1) a second video processor connected to the bus line including: a background image processing section, for generating background image data; and a display control section, connected to the background image generation section and to the first image processing section, for controlling the image data in order to combine the sprite image data and the background image data; and PA1 2) a second video RAM, connected to the second video processor, for storing image information for generating background image data, wherein the background image processing section includes the following 1-3: PA1 1) a first video processor, connected to the bus line, f or controlling the generation of sprite image data ; PA1 2) a first video RAM, connected to the first video processor, for storing image information for generating the sprite image data; PA1 3) a frame buff er for expand ing the sprite image data; and PA1 4) a bit value change means for reading sprite image data expanded in the frame buffer, changing the value of a specified bit of the image data, and then writing the image data back at the same address in the frame buffer, PA1 1) a second video processor, connected to the bus line, for generating background image data; PA1 2) a second video RAM, connected to the second video processor, for storing image information for generating background image data; and PA1 3) a display control section for controlling the image data in order to combine the sprite image data and the background image data, wherein the second video processor includes the following 1 and 2: PA1 coefficient memory for storing coefficient k for one screen on a pixel basis; PA1 a matrix operation circuit for performing the calculation of the formula (1) and the formula (2) to find Xp, Yp, Xs, and Ys of each pixel; and PA1 an AND/OR operation circuit for performing the calculation of the formula (4) using Xp, Yp, Xs, and Ys of the pixel and the coefficient k of the corresponding pixel read from the coefficient memory to find coordinates X and Y. PA1 1 a parameter register for storing signals of rotation matrix parameters A-F supplied from the CPU as well as coordinate data before transformation Px, Py, Pz, Cx, Cy, Cz, Mx, My, and Mz, in which a viewpoint after coordinate transformation (Xp, Yp, Zp) and a specified point on a display screen after coordinate transformation (Xs, Ys, Zs) are represented by the following formulas: ##EQU3## where (Px, Py, Pz) is a viewpoint before coordinate transformation, (Cx, Cy, Cz) is a central point of coordinate transformation, (Sx, Sy, Sz) is a specified point on a display screen before coordinate transformation, and (Mx, My, Mz) is a parallel displacement amount, and the coordinates X and Y of a point on a background screen on which a screen that has been transformed according to the above formulas is displayed are represented by the following formulas: EQU X=k(Xs-Xp)+Xp EQU Y=k(Ys-Yp)+Yp (4) PA1 2 Coefficient memory for storing coefficient k, supplied from the CPU, for each pixel; PA1 3 a matrix operation circuit, connected to the parameter register, for calculating Xp, Yp, Xs, and Ys of each pixel by performing matrix operation of formula (1) and formula (2); and PA1 4 an AND/OR operation circuit for performing the calculation of formula (4) with the use of Xp, Yp, Xs, and Ys of the pixel and the coefficient k of the corresponding pixel read from the coefficient memory to find coordinates X and Y and then generating a read address in the video RAM. PA1 (1) a first terminal for accepting signals from a CPU; PA1 (2) a second terminal for connecting a video RAM in which image information is stored; PA1 (3) a third terminal for outputting image data; and PA1 (4) a display image generation block including the following circuits 1)-5): PA1 (5) a display control block, connected to the display image generation block, for outputting generated image data from the third terminal. PA1 (1) a first terminal for accepting signals from a CPU; PA1 (2) a second terminal for connecting a video RAM where image information is stored; PA1 (3) a third terminal for outputting image data; PA1 (4) a display image generation block including the following circuits 1)-8): PA1 (5) a display control block, connected to the display image generation block, for outputting generated image data from the third terminal.
In the above formulas, k is a perspective coefficient.
By writing the calculated four values into the image processing system, the image processing system adds .DELTA. GX and .DELTA. GY to GXst and GYst, respectively, for each dot to find the scroll screen coordinates.
This calculation requires a large matrix operation circuit including a multiplier. To keep the circuit small, the above calculation is usually performed by a software routine.
In the above technique, the following problems must be solved:
Problem 1: It is difficult to generate a variety of image data without increasing the amount of image memory.!
Today, a video entertainment system must be capable of displaying life-like, complex images. For example, as a variation of a rotation scroll screen, an image produced by rotating an original background image in VRAM clockwise and another image produced by rotating the same original background image counter-clockwise are combined into one background image. In such a case, image data produced by rotating the original data clockwise and image data produced by rotating the original data counter-clockwise are generated separately. Therefore, the video entertainment system must access VRAM to read pattern name data each time image data is generated, and must perform rotation processing for the pattern data of each cell image. Because this method requires two separate VRAM accesses, the access time becomes longer and display data is sometimes not available when it is required.
To solve this problem, two background images--clockwise rotation image and counter-clockwise rotation image--are stored in VRAM beforehand, and the pattern data and pattern name data are read according to the clockwise and counter-clockwise rotation coordinate values. However, it requires more VRAM (i.e., a plurality of physically-independent VRAM areas or an increased amount of VRAM).
Problem 2: The display of the scroll screen increases the CPU load!
As the background image is moved or rotated, the start point of the X and Y coordinate values and the horizontal increments .DELTA. X and .DELTA. Y must be calculated for line by line. Calculation of movement and rotation requires a large CPU overhead. This increases the CPU load, limiting the amount of work the CPU does for other jobs.
Problem 3: A video entertainment system capable to perform the 3-axis rotation of the scroll screen!
In the conventional method for controlling a scroll screen, the increments .DELTA. X and .DELTA. Y of each line on the display screen are constant. Therefore, it is impossible to rotate an image on three axes (X-axis, Y-axis, and Z-axis) at the same time; that is, the background image cannot be tilted during display so that the right end is away from (or near to) the left end. Because of this, the display ability of the scroll screen is limited.
Problem 4: A need for a variety of window displays!
In the conventional window display method used on a video entertainment system, two types of window are available: a rectangle window which is displayed according to two specified points, and a line window which is displayed according to two specified points on a line. However, these windows are simple in shape and, so, it is impossible to display various types of windows.
In view of the problems given above, it is the main object of this invention to provide an image processing system for generating a variety of split screen image data as well as its method and to provide an electronic system having an image processing system which achieves this object.
A first object of this invention is to provide an image processing system for preventing the CPU load from increasing and for creating various split-screen displays through the efficient use of image memory.
A second object of this invention is to provide an image processing system for implementing various types of transformation while keeping the CPU load to a minimum by performing movement transformation processing and/or rotation transformation processing in special operation circuits. A third object of this invention is to provide an image processing system capable of creating highly efficient split-screen display. A forth object of this invention is to provide an image processing system for reducing the time spent in generating display image data.
A fifth object of this invention is to provide an image processing method for preventing the CPU load from increasing and for creating various split-screen displays through the efficient use of image memory. A sixth object of this invention is to provide an electronic system, especially a video entertainment system, for preventing the CPU load from increasing and for creating various split-screen displays through the efficient use of image memory.
A seventh object of this invention is to provide an electronic system, especially a video entertainment system, which have the CPU, two pairs of video processors, and VRAM and which can create various split-screen displays. An eighth object of this invention is to provide an electronic system, suitable for use on a video entertainment system, which generates a sprite image and a background image, for reducing the CPU load and for displaying various split-screen displays through the efficient use of image memory.
A ninth object of this invention is to provide an electronic system, suitable for use on a video entertainment system, which is capable of creating various split screen displays through the efficient use of image memory and window display of any shape. A tenth object of this invention is to provide an electronic system, especially suitable for use on a video entertainment system, which can create a window of any shape.
An eleventh object of this invention is to provide an image processing system which can generate image data as if it was generated through 3-axis rotation and which can reduce the CPU overhead. A twelfth object of this invention is to provide an electronic system, especially suitable suitable for use on a video entertainment system, which can generate image data as if it was created through 3-axis rotation and which can reduce the CPU overhead.
A thirteenth object of this invention is to provide an electronic system which is suitable for a video entertainment system for achieving the above objects and which is provided with a display device.
A fourteenth object of this invention is to provide an image processing system which can achieve above objects and which can be packaged on a semiconductor board. In particular, a fifteenth object of this invention is to provide an image processing system which has color RAM and which achieves the object described in the fourteenth object. A sixteenth object of this invention is to provide an image processing system which achieves the above described objects and which has control registers and window control blocks which can be modified by the CPU. A seventeenth object of this invention is to provide an image processing system which achieves the above objects and whose image processor for generating background images can be packaged on one semiconductor board.
An eighteenth object of this invention is to provide an image processing system which processes rotation processing on the display screen at a high speed and which can react to a video entertainment system player's action in real time. A nineteenth object of this invention is to provide an image processing system which can generate image data as if 3-axis rotation was performed on the image. A twentieth object of this invention is to provide an image processing method for creating various types of screen displays.